Since the year 2003 the police department of san francisco has been reporting crime data. All these data are available to the public and therefore relevant to do data analysis on. Of particular interest for data analysis is the different crime types, the time of incident (both date but also time of day down to the minute) but also the coordinates of the incident (given in latitude/longitude). From this data its possible to look at temporal and spatial trends of different crimes over the last 20+ years. The different categories of crimes include vehicle theft, vandalism, robbery, prostitution and many more.
We have chosen to look at the trends of vehicle thefts since the trend are “unique” compared to the other types of crimes. This will be shown in the following plots.
The temporal trend of vehicle thefts
The first relevant plot is the number of incidents of vehicle theft per year. We ignore 2025 since the we do not have data for that entire year.
Code
import pandas as pd import numpy as npimport matplotlib.pyplot as pltdata=pd.read_csv("C:/NoterDTU/6_semester/Social_data/website_2/s224394.github.io/merged_data.csv")crimes = data[['Category', 'Year']]crimes = crimes[(crimes['Category']=='VEHICLE THEFT') & (crimes['Year']!=2025) ]crime_counts = crimes["Year"].value_counts().sort_index()plt.figure().set_figwidth(7.4)crime_counts.plot(kind="bar",color="indigo",edgecolor="black")ax=plt.gca()ax.set_facecolor("#f5f5f5")plt.ylabel("Number of incidents")plt.xlabel("Year")plt.title("Number of Vehicle thefts per year (2003-2024)")plt.show()
One thing that seems unique is the sudden drop from 2005 to 2006 and onwards. In 2005 the numbers peak at around 17.500 vehicle thefts while the next year it drops by around 10.000 and remains in that range going forward. This is approximately 60% of the crimes that just stopped happening in one year. One would suspect that the police force might have increased the resources for fighting vehicle theft. Another explanation might be that cars are harder to steal today because of increased security measures on the cars.
In the news article “Car Thefts Decrease Statewide” by east bay times from 2007 (Staff 2007) talks aboutWhere the general trend for vehicle theft are on the decline. The reason behind this trend is both the fact that more and more vehicle have implemented alarms, key-coding systems. But also there has also been set up 16 auto-theft task forces. There ways also an increase in the use of so called “bait-cars”. The way “bait-cars” work is by including a hidden gps in the car and making it intentionally easy to steal. The cars are then used to track down the drivers. Since the car thieves often are responsible for stealing more than one car. Catching one criminal might decrease the number of cars being stolen significantly. In 2006 they made 357 arrest with the use of bait-cars. Which might have severely impacted the amount of cars stolen.
We also see an upward trend in vehicle thefts after post-2010 and post-2020. One theory as to why this is the case is related to the two economic crisis. The 2008 financial crisis and the beginning of the covid-19 pandemic in 2020 might have pushed more people into poverty. Which in turn then makes it so more people commits crime in order to survive.
The spatial trends of vehicle theft
The figure and the news article from before suggests that we might be able to see a decrease of crimes if we look at a heatmap of the crimes. One could suspect that the increase in auto-theft task forces might have forced the incidents to move location. In order to look into this we plot the spatial data for vehicle thefts as a heatmap. We look at the incidents for a given month starting january 2003 and going forward.
Code
import foliumfrom folium.plugins import HeatMapWithTimefrom IPython.display import display# Load datadf = pd.read_csv("C:/NoterDTU/6_semester/Social_data/website_2/s224394.github.io/merged_data.csv")# Filter for vehicle thefts between 2003-2007df_filtered = df[(df['Category'] =='VEHICLE THEFT') & (df['Year'].between(2003, 2024))].copy()# Extract relevant columns and drop NAdf_filtered = df_filtered[['Latitude', 'Longitude', 'Month', 'Year']].dropna()# Check for valid coordinatesvalid_coords = df_filtered[ (df_filtered['Latitude'].between(-90, 90)) & (df_filtered['Longitude'].between(-180, 180))]# Define month mapping and ordermonth_mapping = {"January": 1, "February": 2, "March": 3, "April": 4, "May": 5, "June": 6, "July": 7, "August": 8, "September": 9, "October": 10, "November": 11, "December": 12}month_names =list(month_mapping.keys())# Create numerical month columndf_filtered['MonthNum'] = df_filtered['Month'].map(month_mapping)# Sort by year and monthdf_filtered = df_filtered.sort_values(['Year', 'MonthNum'])# Prepare heat data and time indexheat_data = []time_index = []for year inrange(2003, 2025):for month_num inrange(1, 13): month_data = df_filtered[ (df_filtered['Year'] == year) & (df_filtered['MonthNum'] == month_num) ] coords = month_data[['Latitude', 'Longitude']].values.tolist() heat_data.append(coords) time_index.append(f"{month_names[month_num-1]}{year}")# Only create map if we have data# Create base mapbase_map = folium.Map(location=[37.75800, -122.41914], zoom_start=11.5,zoom_control=0,scrollWheelZoom=False,dragging=0)# Add heatmap with timeHeatMapWithTime( heat_data, index=time_index, # Time labels showing month and year auto_play=0, max_opacity=0.5, radius=11, min_opacity=0.1, gradient={0.2: 'blue', 0.4: 'lime', 0.6: 'orange', 0.8: 'red'}, display_index=True, use_local_extrema=1, name="Vehicle Thefts", blur=1).add_to(base_map)# Display mapdisplay(base_map)
Make this Notebook Trusted to load map: File -> Trust Notebook
From the heatmap its clear that a lot of the incidents happen in the north-eastern/eastern part of san francisco. Even after the massive drop of vehicle thefts it still seems to be primarily a given area. This might suggest that there are some factors that make the area more prone to car thefts. The time series heatmap shows the density of crimes dropping when looking at 2003-2005 versus 2006-2024. This is as expected. You can also see the increase from 2010 and 2020 as previously discussed.
Correlation between crimes
In order to determine how unique the trend of vehicle theft are it makes sense to compare the data to the other crimes reported. We choose to look at the the number of incidents per month for a given crime. Afterwards we see how correlated the data for two crimes are. The correlation coefficient for the given plot are also calculated as to give a statistical measure of how correlated the data are. In the plot its possible to choose which crime categories to compare. If you hover over one of the points you get the month of the data and also the number of incidents.
Comparing vehicle theft to the other crimes one sees that there is close to zero correlation between any of the crimes and vehicle theft. The reason behind this is probably the incidents of the years 2003-2005. In most of the plots comparing vehicle data the data from 2003-2005 is completely separate from the rest of the data. This suggest that the sudden drop of vehicle thefts are unique compared all the other crimes. The plots suggest that if we only only looked at the data post-2006 then the correlation would be much greater. One theory as to why this is the case is because of the of focus an battling vehicle theft in 2006. Which lead to a 60% drop in crimes. Where as other crimes such as burglary and vandalism where not focused on. In order to fully determine if the behavior of vehicle theft are unique we also looked at the correlation between some of the other crimes. An example could be robbery and assault which are way more correlated (\(r^2\) value of 0.485) and generally if we compare most of the crimes with the total number of crimes, then they are fairly correlated. Examples being vandalism having \(r^2=0.486\) and larceny/theft having \(r^2=0.700\). Some of this might also be explained in larceny/theft and vandalism playing a bigger part of the crime incidents numbers. Not all types of crimes peak in the same months. But this ones again suggest that vehicle massive drop where unique.
Conclusion
We have in this article looked at three different plots. Where in particularly the temporal and corelation plots suggest that the trend of vehicle thefts incidents are unique compared to the temporal trends of other crimes. This is concluded both because the number of vehicle thefts drop by approximately 60% in one year. But also because the corelation between number of vehicle thefts compared to the incident number are generally close to zero. A news article from 2007 explains that the san francisco police department set up 16 auto-theft task forces and generally caught a lot of car thieves in 2006. This might explain why there was a sudden drop in vehicle theft incidents. This also might explain why the different crimes are not as much correlated with vehicle theft as they are with each other.
